UWSpeech: Speech to Speech Translation for Unwritten Languages

Existing speech to translation systems heavily rely on the text of target language: they usually translate source language either and then synthesize from text, or directly with for auxiliary training. However, those methods cannot be applied unwritten languages, which have no written phoneme available. In this paper, we develop a system named as UWSpeech, converts into discrete tokens converter, translates source-language translator, finally synthesizes an inverter. We propose method called XL-VAE, enhances vector quantized variational autoencoder (VQ-VAE) cross-lingual (XL) recognition, train converter inverter UWSpeech jointly. Experiments Fisher Spanish-English conversation dataset show that outperforms direct VQ-VAE baseline by about 16 10 BLEU points respectively, demonstrate advantages potentials UWSpeech.